The present invention relates to electronic apparatus for communication by the deaf, and, more particularly, to such apparatus which enables the deaf person to communicate through use of sign language.
Deaf people are employed in almost every occupational field. They drive cars, get married, buy homes, and have children, much like everyone else. Because of many inherent communication difficulties, most deaf people are more comfortable when associating with other deaf people. They tend to marry deaf people whom they have met at schools for the deaf or through deaf clubs. Most deaf couples have hearing children who learn sign language early in life to communicate with their parents. Many deaf people tend to have special electronics and telecommunications equipment in their homes. Captioning decoders may be on their televisions, and electrical hook-ups may flash lights to indicate when the baby is crying, the doorbell is ringing, or the alarm clock is going off.
However, deaf persons have substantial difficulties in communicating with persons at remote locations. One technique which is employed utilizes a teletype machine for use by the deaf person to transmit his message and also to receive messages, and the person with whom the deaf person is communicating also has such a teletype machine so that there is an effective connection directly between them. In another method, the deaf person utilizes a teletype machine, but the person who is communicating with the deaf person is in contact with a communications center where a person reads the transmission to the hearing person over the telephone and receives the telephone message from the hearing person and transmits that information on the teletype machine to the deaf person. Obviously, this teletype based system is limited and requires the deaf person to be able to manipulate a teletype machine and to understand effectively the written information which he or she receives on the teletype machine. Processing rapidly received written information is not always effective with those who have been profoundly deaf for extended periods of time. Moreover, a system based upon such teletype transmissions is generally relatively slow.
The widespread availability of personal computers and modems, has enabled direct communication with and between deaf persons having such computers. However, it is still required that the deaf person be able to type effectively and to readily comprehend the written message being received.
Deaf persons generally are well schooled in the use of finger and hand signing to express themselves, and this signing may be coupled with facial expression and/or body motion to modify the words and phrases which are being signed by the hands and to convey emotion. As used herein, "signing motions" include finger and hand motions, body motions, and facial motions and expressions to convey emotions or to modify expressions generated by finger and hand motions. A written message being received on a teletype machine or computer may not convey any emotional content that may have been present in the voice of the person conveying the message.
Profoundly deaf people communicate among themselves by this sign language on a face to face basis, and utilize a Tele-Typewriter (TTY) for telephone communication. The TTY itself leaves much to be desired, since their sign language is a modified syntax of the spoken language, resulting in a smaller vocabulary and lessened ease of reading printed text as a whole (e.g. definite and indefinite articles ["the", "a", "an"] are omitted most of the time and possessives and plurals are not usually distinguished.
When it comes to communication of profoundly deaf persons and normally hearing persons, the problem intensifies. Only a negligible percentage of the non-deaf population is versed in sign language. Thus, some deaf people read lips and utter words similar enough in their vocal resemblance to enable them to be understood. Beyond this tedious and taxing effort, there is virtually no form for such communication except exchanging some written notes or having an interpreter involved.
A number of methods as to how to achieve sign recognition have been proposed in the literature. However, in spite of the apparent detail of such articles, they do not go beyond general suggestions, which fail when tested against the development of enabling technology. Major problems have been impeding the success of such enabling technology.
The Kurokawa et al article entitled "Bi-Directional Transmission Between Sign Language And Japanese For Communication With Deaf-Mute People" Proceedings of the 5th International Conference on Human Computer Interaction, 2, 1109 (1993) described how limited recognition can be achieved of static gestures utilizing electromechanical gloves which are sensor based and Kurokawa digitizes the electromechanical output of sensors. Capturing images with a camera is a well known art, but interpreting such images in a consistent way without relying on the human brain for direct interpretation (i.e., machine interpreted images) has alluded researches. The Rogers article entitled "Proceedings SPIE-The International Society For Optical Engineering: Applications of Artificial Neural Networks", IV, 589 (1993), suggests various approaches which cannot work when tested in a real life situation, such as utilizing infrared for signal interpretation. Unfortunately, one cannot combine the technology of Rogers and Kurokawa to solve the problem because the technologies employed are mutually exclusive. If one uses images as Rogers proposes, one cannot obtain from them the information provided by the sensors of the data gloves of Kurokawa; if one uses Kurokawa's gloves, one cannot utilize the camera images to provide any intelligence, knowledge or information beyond what the sensors in the DataGloves provide. Therefore, a fresh approach to the problem is necessary.
Displaying signed motions presents another challenge. A simple database of all possible signed motions which is an intuitive approach is rather problematic. To create a lucid signing stream, one needs a smooth movement from one word or phrase to another. Otherwise, the signing is jerky at best if not totally unintelligible. Although there may have been suggestions for such a database of signing images, this is not a realistic resolution due to the fact that, for every signed image in the database, one will need to have an enormous amount of connecting movements to other potential gestures, increasing dramatically the size of the database. To select a signing stream, inclusive of all the proper intermediary connecting gestures between previous and current images needed for lucid signing presentation, from such an enormous database puts search algorithms to an unrealistic challenge.
Attempts have also been made to transmit digitized signing motions to a central station as disclosed in Jean-Francois Abramatic et al, U.S. Pat. No. 4,546,383. Even when images are transmitted as proposed by Abramatic et al, the edge detection performed fails to enunciate detail of overlapping hands, or to differentiate between finger spelling and signed motions. All such attempts are restricted by available bandwidth which curtails wide use of such methods.
It is an object of the present invention to provide a novel electronic communication system for use by deaf persons to enable them to communicate by signing.
It is also an object to provide such an electronic communication system wherein the deaf person and the person communicating with the deaf person do so through a central facility containing a translating means for processing elements of digitized image data.
Another object is to provide such a system in which a hearing person may have his speech converted into digitized signing motions which are displayed to the deaf person.
A further object is to provide a unique method utilizing such an electronic communication system to enable communication by and to deaf persons.